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1 .  INTRODUCTION 


This  report  covers  our  efforts  to  creete  a  flexible,  petto* 
no li sod  graphics  workstation  for  the  dec i s i onetke r .  Ve  hove 
designed  4  comprehensive  system  to  be  built  in  Phase  II  of  this 
research.  For  this  phase  we  built  a  prototype  that  included  many 
of  the  important  features  of  the  eventual  system. 

Such  a  workstation  would  assist  the  decisionmaker  by  acting 
as  a  data  analysis  aid.  The  key  elements  of  the  workstation 
include : 

1>  A  large  set  of  formats  tor  the  graphical  display  of  data. 

2)  A  mechanism  for  choosing  formats  to  display  sets  of  data 
from  a  large  database  of  information. 

3)  A  mechanism  for  specifying  patterns  or  events  to  search 
for  in  the  set  of  data  streams;  when  a  pattern  is  found,  it 
triggers  an  action,  which  may  include  the  display  of  relevant 
data. 

This  report  also  describes  preliminary  experiments  that  we 
performed  on  the  prototype  of  this  system.  The  first  set  of 
experiments  seeks  to  show  differential  behavior  produced  by  dif¬ 
fering  display  formats  of  the  same  data.  The  second  set  of 
experiments  studies  subjects*  control  of  a  simple  environment 
that  is  changing  in  real  time  (a  chemical  factory),  where  a 
battery  of  display  aids  are  provided.  These  aids  include 
multiple  display  formats  and  a  programmable  event  monitor 
patterned  after  the  production  systems  familiar  in  AI  research. 

In  both  sets  of  experiments  we  have  shown  correlations 
between  differences  in  display  formats  and  differences  in 
behavior.  However,  these  correlations  are  not  simple.  They 
have  given  us  insight  into  unexpected  complexities  of  behaviour 
and  provide  guidance  tor  further  refinements  of  our  system  and 
for  other  systems  of  data  presentation. 


This  report  contains  three  major  sections: 

1)  the  SUMMARY  OF  RESEARCH,  which  briefly  describes  several 
aspects  of  our  effort; 

2)  TECHNICAL  FINDINGS,  which  elaborates  on  each  of  the 
subsections  in  the  first  section; 

3)  CONCLUSIONS,  which  states  our  feelings  as  to  value  of 
concept  and  asserts  that  we  have  demonstrated  its 
feasibility. 
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2.  SUMMARY  OF  RESEARCH 


2.1.  Design  Rational* 

Making  effective  dtcisiont  rtquirtf  analysing  and  understan- 
ding  data  (which  may  ha  ganaratad  in  raal  tima  or  which  may  ha 
ratriavad  from  a  dattbasa) .  This  raqui rat  baing  abla  to  saa 
patterns  in  numerical  information.  A  graphics  -workstation  is 
just  tha  tool  for  assisting  in  this  process. 

The  most  obvious  function  of  such  a  workstation  would  ba  to 
graphically  encode  tha  data,  thus  making  tha  patterns  evident. 
Additionally,  tha  system  can  make  evident  different  types  of  data 
patterns  if  it  has  facilities  for  graphically  encoding  data  in 
many  ways.  However,  simply  baing  abla  to  display  all  tha  data  is 
not  enough.  With  large  amounts  of  data,  tha  user  will  be  over¬ 
whelmed,  even  if  tha  data  is  represented  compactly.  Therefore, 
we  augment  the  system  with  simple  pattern  detectors  (event  moni¬ 
tors).  These  event  monitors  scan  the  data  for  the  simple  pat¬ 
terns.  When  a  qualifying  pattern  is  detected  the  event  monitor 
triggers  an  action,  which  may  include  the  display  of  relevant 
data,  a  message  to  the  decisionmaker,  or  a  control  response 
(which  may  alter  the  behavior  of  the  system  itself  or  the  system 
that  it  is  monitoring). 

The  user  knows  best  the  types  of  patterns  that  he  is  looking 
for  in  the  data,  so  these  components  should  be  personalised. 
(That  is,  they  must  be  programmable.)  This  means  that  he  should 
be  able  to  specify  what  events  he  is  looking  for  in  the  data  and 
what  is  to  happen  when  those  events  are  detected.  Furthermore, 
he  should  be  able  to  tell  the  system  which  data  to  display  and 
how  and  when  to  do  it.  Finally,  he  should  be  able  to  do  all  this 
with  a  minimum  of  training. 


2.2.  Overall  System  Architecture 

Our  system  would  have  the  following  components:  A  Data 
Collector  (item  •!  in  Figure  1),  which  takes  data  from  a  database 
or  from  a  stream  of  input  data  and  inserts  them  into  a  State 
Variable  Area  (2).  The  State  Variable  Area  contains  the  data  of 
interest  to  the  user.  The  Event  Monitor  (3)  watches  the  data  as 
it  is  collected.  When  one  of  a  set  of  user-defined  conditions 
occurs,  the  event  monitor  executes  an  associated  action,  which 
may  be  a  modification  of  a  state  variable  or  which  may  be  the 
initiation  of  some  display  action.  The  Display  Subsystem  (4) 
executes  this  display  action  by  graphically  encoding  relevant 
data.  The  User  Interface  (5)  allows  (among  other  things)  the 
definition  of  these  event/action  pairs,  which  are  inserted  into 
the  Knowledge  Base  (6).  Further,  the  User  Interface  allows  users 
to  interact  with  the  display  subsystem  to  define  display  actions. 
Finally,  the  system  requires  a  System  Monitor  (7)  to  coordinate 
the  actions  of  all  the  above  components.  (See  detailed  descrip¬ 
tion  in  sect  ion  3 . > 

The  user  would  interact  with  this  system  in  two  modes: 
training  mode  and  running  mode.  Training  mode  is  for 
personalising  the  system.  In  this  mode  the  user  tells  the  system 
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what  types  of  patterns  to  look  for  in  tho  data  and  what  to  do 
whon  thoso  patterns  aro  encountered.  Among  these  “thing*  to  do" 
are  display  actions,  which  the  user  must  also  describe. 
Describing  a  display  action  involves  specifying  what  and  how  data 
are  to  be  displayed.  Once  the  system  has  been  personalised,  it 
is  set  into  running  mode.  Then,  when  interesting  events  occur, 
the  system  takes  the  corresponding  action  (depending  on  the 
importance  of  the  event  and  what  was  on  the  display  already). 

Ve  chose  the  Chromatics  CGC  7900  as  the  device  on  which  to 
implement  a  prototype  of  this  system.  It  is  a  h i gh- r eso 1 u t i on 
(1024  by  768)  bitmap  color  (up  to  256  different  colors)  micro¬ 
computer  with  relatively  sophisticated  software  (including  a 
UNIX-like  operating  system,  Fortran,  and  C) . 


2.3.  Limitations  of  Experimental  Environment 

It  was  impossible  for  us  to  implement  the  complete  system  in 
the  time  allocated,  so  we  decided  to  implement  part  of  the  system 
and  simulate  the  rest.  Then  we  could  expose  users  to  the 
simulated  system  to  test  the  value  of  the  concept. 

In  the  first  experiment,  we  tried  to  simulate  the  per¬ 
sonalised  display  format  aspect  of  the  system  by  having  subjects 
view  the  display  of  the  same  data  in  several  graphical  formats. 

For  the  second  experiment,  we  implemented  the  display  sub¬ 
system  in  the  context  of  the  real-time  control  of  a  primitive 
simulated  chemical  plant.  The  simulated  system  displays  the  state 
of  the  "chemical  plant"  using  a  fixed  set  of  dials.  The  display 
can  be  augmented  with  graphs  of  the  history  of  the  data  displayed 
in  the  dials.  (See  Figures  2  and  3  for  an  example  display  from 

this  simulation.  Figure  2  shows  the  left  half  of  the  display, 
which  contains  the  dials,  and  Figure  3  shows  the  right  half  of  the 
display,  which  contains  the  history  displays.)  Furthermore,  the 
user  can  interactively  define  simple  events  that  the  system  will 
look  for.  When  these  events  occur,  the  user  is  notified.  The 
results  from  these  experiments  are  presented  in  Section  3.2. 

We  believe  that  these  two  experiments  test  very  important 
aspects  of  the  general  design  presented.  They  point  to  the  value 
of  the  general,  flexible,  and  trainable  system  proposed  tor 
Phase  II  research  and  development. 


24.  Display  Subsystem 

The  display  subsystem  primarily  comprises  subroutines  from  a 
system  we  developed  called  DIGR.  These  routines,  which  we  imple¬ 
mented  on  the  Chromatics,  facilitate  the  display  of  data  in  a 
multitude  of  wa y s . 

DIGR  allows  the  user  to  separate  two  parts  of  the  data 
display  task,  specifying  what  to  display  and  specifying  how  to 
display  it.  Furthermore,  the  "how  to  display  it  part"  is  done  in 
such  a  way  as  to  be  as  independent  of  display  format  as  possible. 
This  means  that  changing  display  formats  involves  a  minimum  of 
effort  on  the  part  of  the  interactive  user.  It  simply  requires 
that  he  invoke  one  of  several  display  formatters.  (For  more 
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details,  see  tht  discussion  in  suction  3.1.3.  > 


2.3.  Event  Monitor  Subsystem 

Although  wt  didn't  implement  ths  event  monitor  with  ths  full 
flexibility  that  is  desirable,  ws  describe  it  in  this  report 
because  we  feel  that  it  is  in  important  pert  of  the  eventual 
system.  The  event  monitor  subsystem  comprises  an  event  monitor 
and  an  associated  knowledge  base.  The  knowledge  base  consists  of 
event/action  pairs  (also  called,  cond i t i on/ ac t i on  pairs).  The 
event  monitor  watches  the  incoming  data  as  it  is  put  into  the 
state  variable  area.  It  constantly  checks  this  area  to  see  if 
any  of  the  conditions  (events)  specified  in  the  knowledge  base 
hold  true.  When  they  occur,  the  event  monitor  initiates  the 
corresponding  action,  which  may  be  a  display  action  or  which  may 
involve  setting  another  state  variable.  This,  in  turn,  may  allow 
more  complex  events  to  be  recognised. 

These  event/action  pairs  are  interactively  defined  by  the 
user.  (In  the  simulated  system,  the  user  was  allowed  to  define  a 
logical  tree  of  simple  binary  relations  on  the  chemical  plant's 
state  variables.  For  example,  subjects  could  define  the  fol¬ 
lowing  event:  if  the  temperature  of  Tank  Z  is  greater  than  a 
threshold  X  and  the  heater  for  Tank  Z  is  on  then  tell  me  to  turn 
the  heater  off.)  In  defining  an  action  in  the  full  system  in  the 
future,  the  user  would  have  the  opportunity  to  interact  with  the 
display  subsystem  to  define  a  display  action. 


2.6.  Experimental  Results 

Ve  ran  two  experiments,  which  were  to  test  what  we  consi¬ 
dered  to  be  essential  features  of  the  system.  One  of  the  expert-  I 

ments  was  to  test  the  value  of  flexible  personalised  displays  in 
the  interpretation  of  data;  the  other  was  to  test  this  in  con¬ 
junction  with  an  event  monitor.  Ve  wanted  to  test  the  value  of 

having  a  variety  of  alternatives  when  displaying  data.  Since  the 
display  system  that  we  implemented  did  not  match  our  speed  re¬ 
quirements,  we  tested  this  by  presenting  data  in  a  variety  of 
formats  simultaneously  so  that  subjects  could  "select"  a  display 
quickly  by  simply  moving  his  eyes  to  the  appropriate  position  on 
the  CRT.  The  results  indicate  that  the  subjects  were  more  accu¬ 
rate  when  more  than  one  format  was  available;  although  their  time 
to  respond  increased. 

It  may  turn  out,  upon  further  investigation,  that  this 
simultaneous  presentation  of  alternate  display  formats  would  be 
better  than  a  sequential  presentation.  The  waiting  time  between 
frames  may  disrupt  the  user's  concentration.  In  any  case,  our 
envisioned  system  would  allow  both  simultaneous  and  sequential 
presentations  of  alternate  formats. 

Furthermore,  we  found  that  different  display  formats  were 
suitable  for  different  information  processing  tasks  (no  surprise 
here).  It  we  could  quantity  exactly  what  each  display  was  good 
for,  we  could  put  that  information  into  the  knowledge  base  so  that 
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the  user  could  automatically  bo  prooontod  with  tho  display  that 
woo  moot  appropriate  to  his  task. 

Tho  rosults  word  not  as  clsarcut  in  tho  chomical  plant 
experiment.  Host  notabio  among  tho  findings  was  tho  superiority 
of  having  tho  event  monitor  and  tho  altornato  displays  for  ono  of 
tho  subtasks  (kooping  tho  chomical  plant  prossuro  and  temperature 
within  safety  limits).  Because  of  high  variability  among  the 
subjects,  tho  significance  of  tho  result  was  only  p< . 1 .  Vo  fool 
that  this  variability  would  be  reduced  if  the  subjects  had  had 
more  training  in  the  system  and  if  tho  system  had  been  more  easily 
customised  and  more  flexible. 
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3 .  TECHNICAL  FINDINGS 


3.1.  Details  of  System  Archi tccturi 

In  this  section,  wo  doscribo  in  sort  detail  tho  system  that 
wo  designed.  (Rcaimbtr,  wo  only  implomontod  a  portion  of  this 
systoa  for  this  phase  of  tho  grant . )  Soo  Figure  1  tor  a  block 
diagram  of  tho  systoa  as  wo  envision  it. 


3.1.1.  Data  Collector 

Tho  Data  Collector  subsystoa  is  tho  interface  between  tho 
workstation  and  tho  world  containing  tho  information  to  bo  ana¬ 
lysed.  This  information  may  reside  in  a  database  or  it  aay 
consist  of  readings  from  a  collection  of  sensors.  In  tho  former 
case,  tho  data  collector  would  act  as  a  database  management 
system  <D8MS)  under  the  control  of  the  user  (via  the  system 
monitor).  As  such,  it  could  retrieve  user-requested  data  for 
analysis.  In  the  latter  case,  the  Data  Collector  would  collect 
data  (synchronously  or  as ynchr onous 1 y )  from  user-selected  sensors 
and  store  it  in  the  State  Variable  Storage  Area.  This  latter 
case  is  our  primary  interest;  we  are  designing  the  system  as  if 
it  were  to  be  used  in  a  real-time  process  monitoring  and  control 
system,  which  may  involve  many  data  streams  at  high  input  rates. 


3.1.2.  State  Variable  Storage  Area 

The  State  Variable  Storage  Area  contains  the  information  of 
direct  interest  to  the  user.  In  the  case  where  the  workstation 
is  used  to  monitor  real-time  data,  this  area  would  contain  1)  the 
raw  sensor  data  (with  recent  histories  and/or  time  stamps),  2) 
statistical  data  generated  from  the  raw  data  (such  as,  running 
averages,  convolutions,  cumulative  sums,  derivatives);  3)  event 
flags  and  variables  (generated  and  used  by  the  event  monitor); 
and  4)  action  demon  flags. 

Of  these,  3)  and  4)  require  further  explanation.  Event 
flags  and  variables  provide  a  mechanism  whereby  events  can  trig¬ 
ger  other  events  (rather  than  simply  triggering  a  display  action). 
The  action  associated  with  a  detected  event  may  involve  altering 
one  of  these  event  flags  or  variables.  This,  in  turn,  may  cause 
some  other  event  condition  to  be  true  since  event  conditions  may 
include  these  event  flags  and  variables.  Thus,  e ven t / ac t i ons  may 
be  chained  or  arranged  as  a  logical  AND/OR  tree.  This  is  a  power¬ 
ful  feature;  however,  it  leads  to  the  sequencing  problem  described 
in  the  next  section.  (An  example  of  event/action  pairs  that 

involve  event  variables  is  given  in  section  3.1.4.) 

Action  flags  provide  a  means  for  initiating  display  actions. 
When  the  event  monitor  detects  an  event,  it  triggers  a  display 
action  by  setting  the  corresponding  flag.  However,  the  action 
doesn't  actually  start  until  the  system  monitor,  which  is  conti¬ 
nuously  polling  the  action  flags,  sees  that  one  of  them  has  been 
set.  The  system  monitor  then  initiates  the  appropriate  action 
and  clears  the  action  fag 
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3.1.3,  Event  Monitor 

The  job  that  t ho  ivint  monitor  perform*  can  bo  simply 
stated:  it  watches  tho  stato  variablos  and  looks  (or  cortain 

conditions  (ovonts) ;  whon  a  condition  is  found  to  bo  trut ,  tho 
event  monitor  initiatos  an  associatod  action.  Tho  conditions  and 
thoir  associatod  actions  aro  storod  in  tho  Knowledge  Bait 
doscribod  in  tho  noxt  soction. 

Ho  wo  vo r ,  tho  ovont  monitor  is  moro  coapl icatod  than  this 
description  makes  it  appear.  Timing  is  one  technical  problem, 
because  tho  system  comprises  (conceptually)  parallel  processes; 
data  is  being  collected  while  it  is  being  monitored  while  it  is 
being  displayed.  How  does  the  event  monitor  know  that  it  is  time 
to  look  through  the  state  variables  for  an  event?  What  if  a 
variable  gets  updated  before  it  can  get  to  it?  Sequencing  is 
another  problem,  because  the  condition/action  pairs  are  partially 
ordered  and  because  the  state  variables  include  event  variables. 
Therefore,  sequencing  is  the  problem  of  trying  to  apply  the 
cond i t i on/ ac t i on  rules  in  the  right  order  to  the  state  variables 
in  the  right  order.  While  we  have  not  completely  solved  these 
problems,  they  are  not  intractable. 


3.1.4.  Knowledge  Base 

This  contains  the  condi t ion/act  ion  pairs  (production  rules) 
that  the  event  monitor  uses.  They  are  essentially  if-then  state¬ 
ments,  which  the  event  monitor  repeatedly  executes.  That  is,  the 
Knowledge  Base  comprises  statements  of  the  form  "if  condition  t 
is  true,  then  perform  iction  Y."  The  condition  can  be 
arbitrarily  nested  logical  (AND,  OR,  NOT)  expressions  containing 
event  flags  or  binary  relations  (greater  than,  equal  to,  less 
than,  etc.)  on  state  variables.  An  example:  "IF  the  tank  temper¬ 
ature  is  GREATER  THAN  the  safety  threshold  AND  the  tank's  furnace 
is  on  AND  the  two  previous  conditions  have  been  true  for  10 
minutes  THEN  print  the  message  'something  has  probably  gone  wrong 
with  the  system  that  turns  off  the  furnace.'  " 

Note  that  the  third  part  of  the  condition  involves  an  event 
variable  (see  section  3.1.2.)  that  keeps  track  of  how  long  the 
first  two  conditions  have  been  true.  There  would  have  to  be  in 
the  Knowledge  Base  other  event/action  pairs  that  would  be  respon¬ 
sible  for  maintaining  this  event  variable.  Suppose  we  call  this 
event  variable  DANGER^ DURATI ON .  Then  there  would  have  to  be 

event/action  pairs:  1)  "IF  the  tank  temperature  just  now  became 

GREATER  THAN  the  safety  threshold  AND  the  tank  furnace  is  on  THEN 
set  DANGER^ DURAT 1 ON  to  0  minutes  and  set  the  DANCER_START_TIME  to 
the  current  time";  and  2)  "IF  the  tank  temperature  is  GREATER 
THAN  the  safety  threshold  AND  the  tank  furnace  is  on  THEN  update 
DANGER^ DURATI ON  to  elapsed  time  (current  time 

DANGER_START_TIME> . 

The  example  illustrates  that  the  ordering  of  the  if-then 
statements  is  important,  since  event/action  pair  1)  sets  an  event 
variable  that  2)  uses,  namely,  D  ANG  E  R_DU  R  AT I  ON  and 

DANGER^ 5TART_TIME .  In  fact,  cond i t i on / ac t i on  pairs  are  only 
partially  ordered,  since  some  of  the  condition/action  pairs  are 
completely  independent  of  each  other. 

The  Knowledge  Base  contains  both  pre-defined  "expert"  rules 
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And  user-defined  "personalised1*  rules.  The  latter  rules  are 
inserted  interactively  via  the  User  Interface  and  saved  for  later 
use  in  the  user's  library. 


3.1.3.  Display  Subsystem 

The  Display  Subsystem  graphically  encodes  selected  data  from 
the  State  Variable  Storage  Area.  It  operates  in  two  modes: 

define  mode  and  display  mode.  The  former  mode  is  for  describing 

display  actions  while  the  latter  is  for  executing  them.  (This 
corresponds  to  the  training  and  running  modes  that  we  described 
in  section  2.2.) 

When  the  user  defines  a  display  action,  he  is  really  doing 
two  things;  first,  he  describes  what  he  wants  displayed,  then  he 
describes  how  to  display  it.  The  first  part  is  a  straightforward 
description  of  the  data:  where  it  is  in  the  State  Variable  Stor¬ 
age  Area,  what  it  looks  like,  etc.  The  second  part  involves 

defining  an  abstract  display  and  then  specifying  a  display  for¬ 
matter  that  is  to  turn  that  abstract  display  description  into  a 
concrete  display. 

Vhat  is  an  abstract  display  and  how  does  one  create  it?  An 
abstract  display  is  a  generic  display;  it  has  all  the  elements 
that  you  would  expect  to  find  in  any  display,  regardless  of  the 
graphical  encoding  scheme  that  the  display  used.  (That  is,  these 
elements  are  independent  of  display  format.)  For  instance,  an 
abstract  display  has  a  title,  axes  (scales  with  tick  marks  and 
labels),  axis  labels,  legends,  assorted  flags,  datum/color  asso¬ 
ciations  (which  result  in  the  display  of  the  datum  always  being  a 
certain  color),  and  the  screen  viewport  in  which  it  is  to  appear. 
The  user  creates  the  generic  display  by  simply  giving  it  a  name 
and  specifying  its  parts:  "1  want  a  display,  call  it  X,  of  state 

variables  w  and  i"  ,  "I  want  the  title  to  be  such-and-such",  "I 
want  the  display  to  appear  on  this  part  of  the  screen." 

Once  the  user  has  created  an  abstract  display,  he  can  call  a 
specific  display  formatter  to  turn  that  abstract  display  into  a 
concrete  one.  That  is,  the  display  formatter  graphically  encodes 
the  data  described  in  the  abstract  display.  (This  is  similar  to 
the  concept  of  device  independence,  where  the  abstract  display  is 
like  a  set  of  device  independent  commands  and  the  display  format¬ 
ter  is  like  a  device  driver.)  The  advantage  of  this  approach  is 
that  the  user  can  look  at  the  same  data  in  different  ways  by 
simply  hooking  in  a  different  display  formatter.  Furthermore,  he 
can  devote  his  full  attention  to  figuring  out  what  he  wants  to 
see  without  having  to  simultaneously  figure  out  what  would  be  the 
best  way  to  view  it. 

The  system  will  include  a  large  library  of  (23  to  30) 
predefined  display  formatters.  The  system  further  will  include 
facilities  for  defining  new  display  formatters.  Ve  would  like  to 
make  this  facility  interactive,  such  that  a  non- p r o g r amme r  could 
define  a  new  display  format  on-line.  Ve  believe  that  we  have 
designed  the  system's  data  structures  so  that  this  is  possible. 
Ve  can  do  this  because  we  have  broken  up  the  definition  of  a 
display  formatter  into  several  modules  (for  example,  a  display 
context  layout  module,  a  display  space  allocator,  and  a  graphical 
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data  tncodtr ) .  The  ui«r  could  then  bo  able  to  combine  predefined 
modultt  in  each  of  thoso  categories  to  create  a  new  display 
formatter;  11  one  from  column  A,  one  from  column  B,  etc." 


3.1.6.  User  Interface 

Ve  envision  the  user  interface  as  a  tree  of  menus,  which 
control  all  aspects  of  the  system.  Ideally,  the  menus  would 
appear  on  a  CRT  separate  from  <but  adjacent  to)  the  display  CRT, 
so  that  menus  don't  compete  with  the  displays  for  screen  space. 
Ve  would  like  the  menu  CRT  to  include  a  touch  panel  and  graphics 
capabilities  so  that  the  menus  could  contain  icons  (to  represent 
the  various  display  formats,  for  instance). 

The  menu  tree  will  have  two  najor  branches,  one  for  training 
the  system  and  the  other  for  controlling  its  ezecution.  In 
training  mode,  the  user  will  have  menus  for  1)  selecting  which 
data  is  to  be  placed  in  the  State  Variable  Area  and  how  it  is  to 
be  stored  (with  history,  with  time  stamps,  etc.);  2)  selecting 
statistical  functions  to  be  performed  on  the  data;  3)  defining 
what  events  the  system  should  look  for  in  the  data  and  what 
actions  to  perform  when  the  events  are  encountered;  4)  defining 
display  actions,  which  require  specifying  which  data  from  the 
State  Variable  Area  to  display  and  what  graphical  format  to  use; 
5)  defining  the  priority  level  (importance)  of  actions;  6) 
defining  the  screen  layout. 

In  running  mode,  the  user  should  spend  most  of  his  time 
monitoring  the  screen;  although  interaction  is  possible.  He  may 
trigger  display  actions  independently  of  the  event  monitor;  that 
is,  he  can  interactively  define  and  view  any  data  from  the  State 
Variable  Area.  When  events  are  detected  by  the  event  monitor  and 
a  display  action  is  triggered,  one  of  three  things  may  happen, 
depending  on  the  action's  associated  priority:  1)  the  display 
will  automatically  be  displayed  (possibly  displacing  a  lower 
priority  display  on  the  screen);  2)  the  user  will  be  sent  a 
message  indicating  that  an  event  has  occurred  and  "would  he  be  so 
kind  as  to  notify  the  system  if  he  wishes  to  see  the  associated 
display";  and  3)  the  display  action  is  completely  ignored  by  the 
system.  Which  of  these  three  alternatives  is  taken  depends  on 
priority  thresholds,  which  the  user  can  interactively  set. 

Designing  the  user  interface  first  requires  designing  the 
user's  mental  model  of  the  system.  This  should  be  a  simplified 
but  accurate  version  of  the  "real"  system,  where  complications  in 
the  system  design  are  hidden.  This  mental  model  must  be  taken 
into  account  when  designing  both  the  User  Interface  and  the 
system  as  a  whole.  More  effort  on  engineering  this  user  model 
would  certainly  be  an  important  component  of  any  eventual  system. 


3.1.7.  System  Monitor 

The  System  Monitor  is  the  y«t-to-be*dcsigned  coordinator  of 
all  the  above  modules.  It's  primary  functions  are  to  1)  mediate 
communication  between  the  user  and  the  Knowledge  Base,  Data 
Collector,  Event  Monitor,  and  Display  Subsystems;  2)  to  provide 
for  the  synchronization  of  the  various  modules;  and  3)  to  handle 
the  display  action  ordering  and  management  for  the  Display 


Subsystem. 


3.2.  Multiple  Display  Formats  Experiment 
3.2.1.  Design 

The  first  experiment  allowed  us  to  explore  two  primary 
questions:  Are  multiple  displays  of  the  same  information  more 
useful  than  single  displays?  Are  different  display  formats 
suitable  for  answering  different  kinds  of  questions? 

The  experimental  procedure  focussed  on  the  presentation  of 
16  by  16  arrays  of  numbers  (see  figure  4  for  an  example)  graph¬ 
ically  encoded  in  four  ways:  intensity  encoded  (figure  5),  sur¬ 
face  encoded  (figure  6),  size  encoded  (figure  7  > ,  or  angle  en¬ 
coded  (figure  8).  A  subject,  sitting  at  the  Chromatics  worksta¬ 
tion,  would  simultaneously  be  shown  one,  two,  or  three  of  these 
encodings  of  a  particular  16  by  16  array.  For  each  such 
screening,  the  subject  would  be  asked  such  questions  as,  "How 
many  peaks  (maxima)  are  visible?1*,  "Are  there  any  pockmarks 
(discontinuities)  in  the  display?",  "Which  of  the  four  extreme 
corners  of  the  array  has  the  highest  value  in  it?"  Questions  of 
how  to  define  a  peak,  a  pockmark,  or  a  corner  had  been  addressed 
prior  to  running  the  experiment.  The  subjects'  response  was 
nothing  more  than  pressing  the  appropriately  numbered  key  on  the 
workstation's  keyboard.  The  subjects  had  been  informed  that  both 
accuracy  and  response  time  were  of  interest. 

Fifteen  subjects  completed  the  full  forty  trial  experiment. 
The  order  of  display  and  question  presentation  was  the  same  for 
all  subjects.  The  particular  16  by  16  arrays  and  their  asso¬ 
ciated  displays  were  organized  in  such  a  way  as  to  allow  infer¬ 
ences  about  format  effectiveness,  display  effectiveness,  and 
potential  differences  between  arrays. 


3.2.2.  Results 

The  unreduced  data  from  this  experiment  consists  of  IS 
subjects'  responses  to  two  primary  questions  (“How  many  peaks  are 
there?"  and  "Which  corner  has  the  highest  value?")  over  40 
trials.  There  are  also  two  secondary  questions  ("How  many 
pockmarks  are  in  the  data?"  and  "Was  there  a  clear  maximal 
value?")  asked  of  each  subject,  but  usually  in  only  20  of  the  40 
trials. 

Our  analysis  consists  of  looking  at  the  given  responses  as 
either  correct/ incorrect  or  as  some  deviation  from  correct  and 
then  simply  comparing  the  performance  for  the  different  display 
formats  aud  for  different  numbers  of  displays.  That  is,  we 
wanted  to  know  if  display  type  was  important  and  if  the  number  of 
display  types  displayed  simultaneously  was  important.  Our 
analysis  takes  place  across  subjects  and  usually  depends  on 
straightforward  applications  of  t-test,  ANQVA,  and  other  group 
comparison  techniques. 

Results  tor  the  accuracy  with  which  subjects  count  the 
number  of  local  maxima  in  the  data  clearly  indicate  the  following 
ordering  of  format  types  from  most  to  least  accurate: 
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SurfAct  >>  Intensity  >  Sis*  >>  Angle 


<">>"  aetm  that  the  difference  is  significant  at  the  .OS  level. 

A  single  ">"  means  that  the  result  represents  a  trend  in  the  data 
that  is  not  significant  at  that  level.)  The  primary  result  comes 
from  the  t-test  procedure  and  is  confirmed  by  a  Duncan's  Multiple 
Range  test  and  its  associated  creation  of  homogeneous  subgroups. 
The  relationship  between  the  formats  holds  up  as  one  conditions 
on  particular  images  or  on  the  number  of  formats  in  the  display. 

The  number  of  displays  on  the  screen  at  one  time  also  affect 
performance.  Two  displays  are  better  than  one  <p<.01>  and  three 
displays  are  worse  than  two  (p<.01>.  The  latter  result  is  sur¬ 
prising.  One  would  expect  that  the  more  displays  there  are*  the 
more  accurate  the  subject  would  be*  since  he  would  be  more  likely 
to  find  the  display  that  suited  the  task.  One  explanation  is 
that  the  increase  in  the  display  number  led  to  an  increase  in 
time  pressure.  (The  subject  would  be  worried  about  getting  the 
questions  answered  quickly  and  would  not  look  at  the  alternate 
displays  properly.)  Another  explanation  is  that  increasing  the 
number  of  displays  increased  the  probability  that  one  of  the 
displays  would  be  an  angle  display.  It  seems  that  angle  displays 
are  so  bad  that  there  mere  presence  is  a  confusing  influence; 
some  subjects  did  not  learn  to  ignore  it  when  it  wasn't  appro¬ 
priate.  A  third  explanation  may  be  that  the  resulting  display  com¬ 
plexity  confused  the  user. 

Both  the  “pockmarks"  question  and  the  "highest  corner" 
question  show  that  as  one  changes  the  question  one  should  realize 
that  alternate  displays  are  called  for.  For  instance*  in 
spotting  pockmarks  (discontinuities)  in  the  data*  intensity  and 
surface  encoding  were  better  (p<.05)  than  size  and  angle 
encoding.  For  comparing  widely  separated  data  points  (corners)* 
size  and  angle  encoding  were  better  (p<.05)  than  surface  or 
intensity  displays.  Again,  these  statements  are  based  on  simple 
differences  in  average  performance  as  measured  by  t-test* 
Duncan's  Multiple  Range  and  related  conditional  tests  on 

subpopu 1  a  t i ons . 


3.3.  Chemical  Plant  Experiment 

3.3.1.  Description  of  simulated  system 

The  chemical  factory  simulation  provides  an  adequately  rich 
and  complex  environment  in  which  experimental  subjects  are  asked 
to  maximize  the  output  of  the  factory  while  keeping  reaction 
temperatures  and  pressures  within  safe  limits.  The  factory  con¬ 
sists  of  three  reaction  vessels  labeled  X,Y,  and  Z.  For  each 
reaction  vessel  the  subject  must  monitor  the  temperature  and 
pressure  given  a  minimum,  maximum,  and  optimum  temperature  and 
pressure.  The  subject  controls  the  temperature  by  activating  and 
deactivating  a  heater  associated  with  each  tank.  Similarly*  the 
subject  controls  the  pressure  in  the  tank  by  opening  or  closing 
either  one  of  two  feedstock  valves  or  by  opening  or  closing  the 
product  output  valve.  Figure  2  shows  the  arrangement  of  the 
three  tanks  in  the  factory.  The  top  two  tanks  (X  and  Y)  produce 
the  two  feedstocks  needed  for  bottom  tank  (Z). 


The  Uetory  model  d«{  intt  the  r*4Ction  rtt»  in  each  vessel 
According  to  tho  Amount  of  the  food  chomici Ic  available  to  react* 
tho  itortgo  fpACo  AVAiUbii  for  tho  product  chemical*  and  tho 
curront  distanco  from  tho  optimum  tomporaturo  and  prooouro  in  tho 
roaction  vossol.  Tho  tomporaturo  in  tho  tank  la  incroasod  if  tho 
hoator  is  on  and  docroasod  by  hoat  loss  to  tho  environment.  Tho 
roaction  may  bo  oithor  ondothormie  or  exothermic  and  affocts  tho 
tomporaturo  in  proportion  to  tho  roaction  rato.  Tho  prooouro  io 
simply  proportional  to  tho  tomporaturo  and  tho  amount  of  ompty 
opaco  in  tho  tank.  Tho  rato  of  chomical  flow  into  and  out  of  tho 
tanks  is  proportional  to  tho  foodotock  prosouro  and  tho  valve 
settings. 

In  ordor  to  holp  tho  usor  control  this  environment  subjects 
were  sometimes  given  a  set  of  nine  alternative  displays  which 
provided  historical  information  about  tho  various  tank  states 
while  at  other  times  subjects  wore  given  a  primitive  event 
monitor  to  holp  monitor  the  factory  state.  Tho  left  side  of 
figure  2  shows  the  complete  state  of  tho  chemical  factory  at  a 
given  instant.  The  displays  on  tho  right  side  of  that  figure 
should  help  the  user  to  understand  how  tho  system  variables  are 
changing  over  time  and  how  the  variables  are  related  to  each 
other.  The  event  monitor  should  remind  tho  usor  when  critical 
events  happen  while  the  subject  is  attending  to  another  portion 
of  the  factory  or  display. 

Tho  usor  can  interactively  define  tho  ovont/action  pairs. 
When  a  specified  event  is  true  or  becomes  true*  the  system  dis¬ 
plays  a  user-defined  message  (the  only  typo  of  action  allowed  in 
this  simulation).  If  the  message  is  displayed  only  when  tho  event 
becomes  true  tho  event  is  referred  to  as  a  triggered  event.  In  tho 
chemical  factory  tho  subject  could  define  triggered  events  to 
remind  him  to  turn  on  and  off  tho  heaters  as  soon  as  tho  tank 
tomporaturo  deviatod  from  the  optimum  by  a  specified  amount.  An 
event  that  displays  its  associated  message  as  long  as  tho  event  is 
true  is  called  a  continuous  event.  Tho  subject  could  define  this 
kind  of  event  to  continuously  remind  him  that  tho  pressure  in  a 
reactor  was  too  high  (since  this  might  damage  tho  factory).  Tho 
events  can  be  simple  relations  between  a  tank  state  and  a 
constant*  such  as,  "Tank  X  temperature  <  .2"  or  they  can  be  more 

complicated,  such  as  in  the  ovent/action  pair:  "if  the  temperature 
in  tank  X  >  .7  and  the  tank  X  heater  is  on  then  generate  tho 

message  "Turn  off  Tank  X  heater". 


3.3.2.  Design 

Vo  asked  each  subject  to  spend  app r o a ima t e 1 y  one  hour  famil¬ 
iarising  himself  with  the  keyboard  responses*  tho  visual  dis¬ 
plays*  and  the  basic  workings  of  our  simple  event  monitor  editor. 
Subjects  accomplished  this  by  reading  a  set  of  instructions* 
asking  questions  of  the  experimenter*  and*  most  importantly,  by 
running  the  system  several  times.  (The  subjects  found  the  system 
tun  to  play  with*  to  tho  extent  that  some  even  asked  to  play  it 
outside  of  the  experimental  context.  Therefore,  wo  refer  to  the 
chemical  plant  control  simulation  as  "the  game.") 

During  this  training  period*  the  subjects  tried  three  dif¬ 
ferent  versions  of  the  system:  one  that  had  only  the  standard 
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dial  display;  on#  that  had  tha  dial  display  and  an  altsrnats 
history  display  s imul t anaous 1 y  on  tha  scraan;  and  ona  that  had 
tha  standard  display  with  an  avant  noni tor/«di tor  available.  Tha 
subjicts  also  triad  tha  game  at  different  spaadt ,  so  that  wi 
would  ba  abla  to  astimata  tima  prassura  tf  facts .  Va  told  all 
tha  subjects  what  tha  minimum,  maximum,  and  idaal  sattings  wara 
tor  tha  tamparatura  and  prassura  in  aach  of  tha  thraa  vats. 
Thasa  sattings  wara  also  incorporatad  into  dials  and  color-coded 
so  as  to  ba  aasily  d i s t ingui shad .  Optimum  valuas  wara 
raprasantad  by  graan  linas,  whila  tha  minima  and  maxima  wara 
raprasantad  by  rad  linas.  (In  soma  varsions  of  this  raport,  tha 
raproduction  of  tha  display  in  figuras  2  and  3  is  in  black  and 
whita,  so  tha  raadar  may  axpaet  to  ba  unclaar  as  to  which  1 i na  is 
which.  Rast  assurad  tha  subjacts  suf farad  no  such  confusion.) 
Va  also  told  tham  what  tha  proportions  of  tha  chamicals  should  ba 
for  optimal  performance.  Tha i r  goal  was  to  stay  within  tha 
minimum-maximum  range  whila  trying  to  maximize  thair  output. 
They  could  use  tha  “Danger  Count"  displayed  in  tha  lower  left 
corner  of  tha  display  (sea  figure  2)  as  a  measure  of  how  wall 
they  wara  keeping  tha  system  below  tha  maximum  prassura  and 
tamparatura  limits. 

After  tha  subjacts  wara  familiar  with  tha  system,  wa  asked 
tham  to  play  six  games  of  100  iterations  aach.  Each  of  thasa 
simulations  ran  at  a  spaed  of  either  3.3  seconds  par  iteration  or 
7.0  seconds  par  iteration.  Each  simulation  had  either  tha  stan¬ 
dard  <s)  display,  tha  alternate  (a)  plus  tha  standard  display, 
and  tha  avant  moni t o r / ad i t o r  (a)  plus  tha  standard  display.  So 
aach  subject  ran  aach  type  of  simulation  at  aach  spaed.  Va  gave 
aach  subject  a  particular  ordering  of  types  and  times  (for  in¬ 
stance,  a-3.3,  s-7.0,  e-3.5,  a-7.0,  s-3.5,  e-7 . 0  )  so  as  to 
control  tha  sources  of  variation  and  to  allow  for  unconfounded 
i n  f a  r anca . 


3.3.3.  Results 

Each  subject  generated  600  linas  of  data,  100  linas  par 

gama,  where  aach  line  gave  a  snapshot  of  tha  tha  chemical 
factory’s  status  at  aach  of  tha  100  iterations  in  tha  gama. 

Initially,  wa  focussed  our  data  reduction  and  analysis  on  tha 
following  variables  for  aach  of  tha  six  runs  of  aach  subject: 
total  danger  counts  (a  measure  of  how  often  tha  subject  was  above 
tha  upper  limit  of  tamparatura  or  prassura  in  a  tank),  total  out- 
of-bounds  count,  and  total  output  (how  much  product  chemical  tha 
subject  produced).  Va  explored  tha  behavior  of  thasa  variables 
in  aggregate,  by  parson,  by  trial  number  (1  through  6),  by  type 

of  trial  (s,  a,  or  a),  and  by  interactions  formed  from  thasa 

variables  (for  instance,  output  as  a  function  of  both  parson  and 
trial  type) . 

Tha  results  are  satisfying,  but,  as  wi th  many  results  in 
human  factors  studies,  they  compete  for  attention  with  tha 
overriding  result  that  there  is  more  natural  variation  between 
people  than  there  is  induced  variation  between  treatments.  Our 
sample  of  eight  individuals  exhibited  a  variety  of  responses  to 
various  displays  ranging  from  total  indifference  to  soph i s t i ca t ad 
usage.  Soma  examples  from  observing  subjacts  during  thair 
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training  and  trial  periods: 

*  Soma  subjects  immediately  took  advantaga  of  tha  altarnata 
tiaa  history  displays  to  spot  tranda  in  taaparatura  and 
pressure,  whila  othar  subjects  found  thaaa  displays 
confusing,  uaalass,  and  aainantly  ignorable. 

*  Vhila  all  subtacts  aastarad  tha  avant  moni tor / ad i t or ,  it 
was  dear  that  soma  iamadiataly  turned  to  it  with  long  lists 
of  avants  to  ba  warnad  of,  but  othars  took  many  trials  to 
saa  tha  uaafulnasa  of  even  a  maagar  list  of  avants. 

*  Va  purpoaaly  gava  tha  subjects  a  modastly  ambiguous  task 
--  "masimise  production,  but  stay  in  bounds11  —  but  soma 
wara  far  more  sansitiva  to  production,  whila  othars  focussed 
on  maintaining  all  subsystans  within  prascribad  bounds. 

Nonathalass,  tha  aasantial  massaga  is  that  tha  typa  of 
display,  tha  langth  of  tima  for  decision,  tha  prasanca  of  a 
trainable  avant  monitor,  and  practice  maka  a  significant 
diffaranca  in  tha  number  of  dangar  counts  racordad  and  in  tha 
amount  of  tima  spant  out  of  bounds. 

However,  tha  story  is  different  whan  wa  look  at  tha  total 
amount  of  output  chemical  produced:  most  of  these  design 

variables  have  no  noticeable  affect.  Two  characteristics  of  tha 
subjects  maka  analysis  difficult  and  highlight  the  value  of  a 
more  formal  training  period;  namely,  there  are  both  vary  consis¬ 
tent  and  vary  inconsistent  individuals  in  our  group  of  eight  and 
avan  in  tha  consistent  groups  we  sea  large  differences  between  a 
subject's  performances. 

To  summarise  tha  results: 

*  The  display  has  an  affect  on  danger  counts,  with  avant 

monitor  (e>  and  altarnata  display  (a)  outperforming  tha 
standard  display  (s).  <  p  <  . 1 > 

*  Tha  display  typa  does  not  show  an  affect  on  total  output. 

*  Increased  tima  per  iteration  (3.5  seconds  vs.  7.0  seconds) 
modastly  increases  output  (.l<p(.2>  and  substantially  de¬ 
creases  danger  counts  <p<.01). 

*  The  trial  number  (1  through  4)  had  no  affect  on  output 
amount,  but  it  has  a  marked  affect  in  decreasing  tha 
dangar  count.  (p<.01,  mean  dangar  counts  montonically  dec- 
with  trial  count  from  33  to  13.)  All  this  means  is  that 
practice  helped. 

*  In  particular,  the  second  trial  of  the  event  monitor  has  a 
sharp  drop  in  danger  counts,  as  does  the  second  trial  of  tha 
standard  display.  Interestingly,  there  is  no  such 
difference  for  tha  visually  richer  altarnata  display. 

*  Subjects  are  far  more  different  from  one  another  than  they 
are  from  treatment  to  treatment,  both  in  terms  of  average 
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icorts  tnd  in  terms  of  level  of  vtrittion  exhibited. 


*  Vo  fool  that  sobo  of  thooo  differences  among  subjects 
would  have  boon  roducod  if  the  subjects  hod  boon  required  to 
spond  moro  timo  looming  tho  system. 

Eoch  of  thoso  rosults  has  booring  on  furthor  tooting  ond 
exploration  of  tho  o f f oc t i vonooo  of  flexible,  individualised 
grophicol  systems.  It  io  particularly  in  tho  evidence  of  wido 
difforoneos  among  subjocto  that  wo  soo  tho  value  of  cuotomisoblo 
displays  ond  event  monitors  in  roducing  both  ho t o r ogono i t y  Cdif- 
foroneos  among  subjocto*  overages)  and  ho t or oscidoi t ic i ty  <dif- 
foroncos  among  subjects*  xariation  around  their  averages).  In 
sum*  tho  system  wo  implemented  was  not  as  pt r sona 1 i tab  I o  as  wo 
would  like.  Vo  fool  that  future  experimentation  on  tho  desired 
system  (whoso  design  wo  haxo  laid  out  above)  would  show  that  tho 
wido  variation  among  subjects  that  we  found  would  bo  roducod 
after  they  had  been  properly  trained  in  its  use. 


4.  CONCLUSIONS 


V*  feel  that  it  is  self-evident  that  a  personal i ttd  graphics 
workstation  would  be  a  powerful  assistant  to  ths  decisionaaker . 
This  would  be  due  to  ths  following  ksy  features  of  ths  system  as 
ws  havs  dssignsd  it:  a  flssibls  display  systsa  capabls  of  dis¬ 
playing  data  in  a  wids  varisty  of  graphical  foraats,  and  an  event 
monitor  that  would  scan  input  data  for  ussr-dsf insd  patterns. 

Although  ws  couldn't  test  ths  complete  system,  ws  tried  to 
isolate  the  two  features  described  above  and  test  their  useful¬ 
ness.  Ve  verified  that  different  display  formats  were  differen¬ 
tially  suited  to  different  information  tasks.  This  would  vali¬ 
date  the  desirability  of  the  first  feature,  the  flexible  display 
system.  While  the  results  in  the  chemical  plant  were  not  as 
clearcut,  we  feel  that  they  indicate  that  the  event  monitor  would 
also  be  a  powerful  addition  to  any  system  that  was  to  assist  in 
the  analysis  and  understanding  of  large  amounts  of  data.  Also, 
the  variability  we  saw  in  subjects1  performances  in  the  chemical 
plant  experiment  underscored  the  need  for  the  system  being 
pe  r  sona 1 i sab  1 e . 

Ve  have  presented  a  viable  design  for  such  a  personal isable. 
intelligent  graphics  workstation.  Ve  feel  that  we  are  capable  of 
imp  1 emen t i ng  it. 
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Figure  t.  Block  Dicgrcm  of  the  Envisioned  System.  <The 
connection  from  System  Monitor  to  Dote  Collector  is  not 
shown.  The  numbers  shown  ere  the  numbers  of  the  components 
described  in  section  2.2.) 
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Figure  3.  The  right  of  the  display  (ret  the  viriion  of 
eheeictl  simulation  the!  hed  the  history  displays  of  key  s 
variables.  (Note  that  in  the  actual  display  the  lines  were 
color  so  as  to  natch  those  of  the  left  half  so  subjects  had 
difficult?  figuring  out  which  lines  in  a  graph  corresponded 
which  state  variable.) 
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Ftgvrc  4.  Citapli  of  4  14  by  14  «rray  of  numbers  that  was 
used  in  Experiment  II  --  tho  Multiple  Foraats  Eipar iatnt . 
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Intinstty  tncoding  data  in  Figure 
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Figure  6.  Parallel  projection  of  a  3-D  surface  that  tncodts 
tha  data  values  (roa  figure  4.  The  peaks  correspond  to 
local  aatlaa  in  the  data. 
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Figure  7.  Sics  encoding  of  the  data  in  Figur 
fquarn  corrtfpond  to  Urgtr  data  values. 
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